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Background: Multidisciplinary team (MDT) working is well established as the foundation 
for providing cancer services in the UK and elsewhere. A core activity is the weekly meet- 
ing (or case conference/tumor boards) where the treatment recommendations for individual 
patients are agreed. Evidence suggests that the quality of team working varies across cancer 
teams, and this may impact negatively on the decision-making process, and ultimately patient 
care. Feedback on performance by expert observers may improve performance, but can be 
resource-intensive to implement. This proof of concept study sought to: develop a structured 
observational assessment tool for use by peers (managers or clinicians from the local workforce) 
and explore its usability; assess the feasibility of the principle of observational assessment by 
peers; and explore the views of MDT members and observers about the utility of feedback from 
observational assessment. 

Methods: For tool development, the content was informed by national clinical consensus 
recommendations for best practice in cancer MDTs and developed in collaboration with an expert 
steering group. It consisted of ten subdomains of team working observable in MDT meetings 
that were rated on a 10-point scale (very poor to very good). For observational assessment, 
a total of 19 peer observers used the tool (assessing performance in 20 cancer teams from four 
hospitals). For evaluation, telephone interviews with 64 team members and all peer observers 
were analyzed thematically. 

Results: The tool was easy to use and areas for refinement were identified. Peer observers 
were identified and most indicated that undertaking observation was feasible. MDT members 
generally reported that observational assessment and feedback was useful, with the potential 
to facilitate improvements in team working. 

Conclusion: This study suggests that observation and feedback by peers may provide a feasible 
and acceptable approach to enhance MDT performance. Further tool refinement and validation 
is required. 

Keywords: cancer, multidisciplinary team, team working, observational assessment 

Introduction 

Cancer multidisciplinary teams (MDTs) are well established in many countries as 
being central to the management and delivery of care.' At the core of the formalized 
structure in the UK (and elsewhere) is the weekly MDT meeting (MDM), sometimes 
known as a multidisciplinary tumor board or case conference in other countries.' This 
brings together cancer health professionals (which may include surgical, medical, and 
diagnostic persormel, and nursing specialists) to discuss options for the management 
of individual cancer patients. The MDM, in best practice, immediately precedes the 
outpatient consultation with patients who were reviewed in the MDM. 
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Although there is accumulating evidence of the benefit 
of MDT working,''^ including an association with improved 
survival and reduced variation in survival,^"^ there is also 
evidence of variability in operational performance and the 
decision-making processes of MDMs.' " A key role of 
specialist cancer nurses in MDMs is to facilitate inclusion 
of the patient's holistic needs and preferences for treatment 
in decision-making; however, the evidence suggests that 
decisions in cancer MDMs are often driven by physicians 
on the basis of biomedical rather than patient-centered 
information and that nurses have little involvement in case 
discussions,'-'^ " despite them being well placed to add holis- 
tic information that impacts greatly on a patient's treatment 
choice. Poor quality discussions in MDMs, in particular 
failure to consider all relevent information, may lead to 
recommendations that are not implemented in practice'*"" 
and/or delays in treatment." '** 

MDMs are an expensive resource." In terms of staff 
time alone, it is estimated that they cost the UK National 
Health Service around £100 million a year for preparation 
and attendance.' In an attempt to improve standards and 
reduce variations in performance, all cancer MDTs in the 
UK imdergo a mandatory "peer review" program involving 
self-assessment and, in some cases, additional review by 
a panel of independent health professionals and service 
users, to assess their compliance with national tumor- 
specific guidelines." However, this formalized process does 
not involve assessment of team working within the MDM 
(eg, leadership/chairing, decision-making processes, patient- 
centeredness, administration, and organization of meetings), 
but concentrates on standards such as patient pathways, 
timelines, access, and other auditable elements. 

Observational case studies of cancer MDMs have 
provided insights into the processes involved in MDT 
decision-making, '^•^° but such methods tend to be time con- 
suming and require methodological expertise.^' Structured 
observational assessment and feedback have proved a use- 
ful technique to help drive improvements in the way health 
care teams work together, for example, to improve patient 
safety during surgical procedures'^ and in anesthesia.'^ This 
approach allows for team work and communication to be 
measured in real life or simulated scenarios, rather than based 
on self-assessment by MDT members. Observation can help 
to elucidate areas where performance could be improved, that 
MDT members may not have been aware of themselves.'"* 

Two observational tools have been developed specifically 
to assess cancer MDM performance, ie, the Metric for the 
Observation of Decision-making,'" which enables assessment 



of discussion content and MDT member contribution, and the 
Observational Assessment Rating Scale,'' which also includes 
assessments of the quality of team working, patient-centered 
clinical decision-making, infrastructure, and organization of 
the meeting. The incorporation of assessment and feedback 
by peers into routine clinical practice may be beneficial and 
a cost-effective intervention for encouraging health profes- 
sional development,'^ but the need for more formalized 
mechanisms (eg, standardized processes, measurable quality 
standards) to facilitate this has been recognized." 

The expense of MDMs, coupled with their influence 
on clinical outcomes, highlights the need to ensure that 
they are functioning optimally. A first step towards this is 
having appropriate methods of assessment that are feasible 
to implement, acceptable to MDT members, and have the 
potential to facilitate improvements in MDM functioning. To 
our knowledge, no research has investigated: the feasibility 
of integrating observational assessment into routine clinical 
practice in cancer MDTs; whether the current workforce has 
the skills to provide useful feedback without being exten- 
sively trained and the capacity to undertake such assessments; 
or if MDT members will find assessment and feedback from 
peers acceptable and useful. Therefore, this proof of concept 
study aimed to; develop a structured peer observational 
assessment tool and explore its usability; explore whether 
the principle of observational assessment by peers is feasible; 
and explore the views of MDT members and observers about 
the utility of feedback from observational assessment. 

Materials and methods 

Design and organizational context 

A qualitative proof of concept study was designed to explore 
observational assessment of cancer MDT meetings by peers 
(Figure 1). Proof of concept (or principle) studies allow for 
the investigation of feasibility of a concept at an early, less 
costly, stage in the research process before developing a 
prototype in later stages. 

A convenience sample of twenty volunteer MDTs within 
four National Health Service hospital trusts that provide 
secondary health services (hereafter referred to as hospitals) 
were recruited (Table 1). Two of the hospital sites were in 
the Midlands, one was in the south of England, and one was 
in London. The hospitals served local populations ranging 
from approximately 750,000 to 2.5 million people. Two were 
imiversity/teaching hospitals. The 20 MDTs represented ten 
different tumor types (head and neck, urology, colorectal, 
lung, gynecological, skin, breast, hepatobiliary, sarcoma, and 
upper gastrointestinal) and included three specialist MDTs 
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Tool development 



Framework for tool development: 
characteristics of an effective MDT^^ 
• Based on survey of over 2000 MDT 
members 

• Identifies "17 sub-domains" for effective 
cancer team w/orking 



Expert steering group and research group 
discussion/consider wider evidence 



Consensus on content 

• 10 sub-domains agreed as observable 

• Translated by research group into tool 

content/format 



Testing proof of concept 



• 4 hospitals, 5 cancer MDTs at each 
hospital participate 

• A meeting for each MDT is assessed 
by peer observer using the tool 

• Feedback from observation is sent to 
all MDT members 



Evaluation to explore: 

• usability of the tool 

• the feasibility of 
observational assessment 

• the utility of 
observational feedback 



Figure I Study design. 

Abbreviation: MDT, multidisciplinary team. 



■ Telephone interviews with 64 MDT 
members and 1 9 observers 
(data collection} 

' Content analysis to identify themes 
(data analysis) 



Table I Profile of hospital trusts, multidisciplinary teanns, and 
time to complete peer observational assessment (minutes) 



Hospital 


Profile of trusts' 


Tumor specialty of MDT 


trust 


and region 


(number of members)'" 


Trust A 


• University hospital 


Head and neck (3 1 ) 




• Midlands, mix of urban 


Colorectal (28) 




and rural areas 


Skin (10) 






Urology (33) 






Lung (S3) 


Trust B 


• General hospital 


Colorectal (17) 




• South England, mix of 


Urology (IS) 




urban and rural areas 


Breast (14) 






Head and neck (24) 






Gynecological (20) 


Trust C 


• General hospital 


Urology (29) 




• Midlands, urban area 


Sarcoma (33) 






Head and neck (S2) 






Hepatobiliary (36) 






Lung (29) 


Trust D 


• University hospital 


Upper gastrointestinal (3S) 




• London, urban area 


Head and neck (47) 






Colorectal (19) 






Gynecological (29) 






Urology (S7) 



Notes; ''All trusts provide a range of cancer services including specialized treatments; 
''all MDTs included surgeons, oncologists, radiologists, histopathologists, clinical 
nurse specialists and other nurses, MDT coordinators (administrators). Some 
MDTs also included palliative care specialists, others physicians, and allied health 
professionals. 

Abbreviation: MDT, multidisciplinary team. 



(ie, MDTs that accept referrals from other hospitals). Five of 
the MDTs used videoconferencing to allow members from 
multiple sites to participate in the MDM. 

Development of the observational tool 

A framework for assessment of team performance in 
cancer MDMs is provided by a document entitled "The 
Characteristics of an Effective MDT".^** Underpinned by a 
survey completed by over 2,000 cancer MDT members in the 
UK, this document contains nearly 1 00 recommendations for 
effective cancer team working organized under 1 7 domains, 
many of which relate to the core MDT meeting function of 
cancer MDT working. We have previously used this document 
as a framework to underpin a self-assessment questionnaire, 
ie, the Team Evaluation and Assessment Measure,^' for MDT 
members to self-assess their performance, and it provided 
the foundation for the design of the observational tool used 
in this study. 

The appropriateness of each of the 17 subdomains of 
team working^^ for observational assessment was deter- 
mined by obtaining views from a national MDT develop- 
ment steering group (consisting of clinical and academic 
cancer leaders, formed to guide the study). In total, ten 
sub-domains were identified to contain elements that would 
be observable in cancer MDMs. These were: attendance at 
MDT meetings, leadership of the MDT and chairing in MDT 
meetings, team working and culture, personal development 
and training (eg, whether members use the meeting as an 
opportunity to share learning and best practice), physical 
environment of the meeting venue, technology and equip- 
ment available for use in MDT meetings, organization and 
administration during meetings, post-meeting coordination 
of services (eg, the clarity of "next steps" in the meeting 
discussion), patient-centered care, and clinical decision- 
making processes (Figure 1). In addition, the frequency of 
distractions (eg, cell phone calls, other conversations) that 
occur during the meeting are recorded. These areas were 
translated into the tool content by the authors. In the result- 
ing tool, observers rated the performance of the MDT in 
relation to each domain on a 10-point scale (1, very poor; 
10 very good; see Figure 2). Brief descriptions of a "very 
good" rating for each of the ten domains were provided 
based on the recommendations in "The Characteristics of 
an Effective MDT".^^ Observers were also asked to provide 
comments to support their ratings, and to provide a free-text 
description of their overall impression of the meeting. They 
were also asked to record the frequency of the different 
types of distractions. 
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Instructions: Please score the team's performance on each of the 10 Characteristics below. The scale is from 1 (very poor 
performance) to 10 (very good performance) - please do use the full range of the scale if appropriate 


CHARACTERISTIC 

(below are abbreviated descriptions of a '10' score) 


SCORE 1-10 

(1= very poor 
10= very good) 


COMMENTS 


Attendance at MDT meeting: Core and extended members attend for all cases 
relevant to them; Decisions are only made when meeting is quorate; Clinician who has 
met the patient is present at the meeting, etc. 






Leadership and chairing in IVIDT meetings: The Chair is a skilled manager of people, 
discussions/decisions and time; The Chair ensures decisions/action plans are evidence- 
based, patient-centered and clear, etc. 






Teamworking and culture: eg. The team has agreed standards for acceptable team 
behavior including respect, equality, openness of discussion, and conflict management, etc 






Personal development and training: eg. The MDT fulfils a teaching and training role 
both for members and beyond the team, etc. 






Physical environment of the meeting venue: There is a dedicated, appropriately 
sized/furnished/equipped room for MDT meetings, etc. 






Technology and equipment available for use in MDT meetings: Meeting room has 
equipment for displaying to whole team radiological images, pathological information, 
databases and input from members at other sites; Equipment is of an appropriate 
standard and reliable, etc. 






Organization and administration during meetings: It is clear who has brought each 
patient to the meeting and why; Locally agreed minimum dataset of info is presented for 
each case; All relevant info (notes, test results/images/samples, appointment dates) is 
available at (or reviewed prior to) meeting, etc. 






Post-meeting co-ordination of services: Processes are in place to ensure MDT 
recommendations are communicated to relevant parties (in an appropriate and timely 
manner), implemented and tracked, etc. 






Patient-centered care: Someone who has met the patient presents their views/ 
preferences/holistic needs; Someone is named responsible for communicating MDT 
outcome to patient, identifying a key worker for patient, etc. 






Clinical decision-making processes: Locally agreed minimum dataset of 
information is available; If any key information is known or feared missing this is 
documented and case deferred if necessary; MDT considers all clinically appropriate 
treatment options, etc. 







Figure 2 Abbreviated version of the prototype Observational Tool ©. 

Note: © Green Cross Medical Ltd. Adapted v/ith permission from Green Cross Medical Ltd. 
Abbreviation: MDT, multidisciplinary team. 



Procedure 

Cancer service managers were asked to identify clini- 
cians (defined here as senior physicians, nurses, or allied 
health professionals) or managers within the hospital 
who were familiar with cancer MDT working and would 
be able to provide constructive feedback to the MDTs. 
These nominated peer observers were provided with the 
prototype observational tool, including integral instruc- 
tions for its use, and a copy of "The Characteristics of an 
Effective MDT".^'* 

Meetings were either observed in vivo or videorecorded 
and viewed by observers at a later date (to allow observer 
anonymity). MDT members were asked to maintain 
patient anonymity by allocating ID numbers to patients. 
Assessments were completed using the observational tool 



and sent to MDT members approximately 5 weeks after the 
observed meeting had taken place. 

Data collection 

Brief, semistructured telephone interviews, using a topic 
guide informed by the study aims, were held with the lead for 
each MDT (n=20), up to three other MDT members (purpo- 
sively selected to represent a range of disciplines, n=44), and 
all peer observers (n= 1 9). Interviews were undertaken by two 
experienced mixed methods researchers (JH and KB) with 
5-10 years' experience in conducting research interviews. 

Data analysis 

Interviews were analyzed using thematic content analysis.'" 
An initial coding scheme was developed (by CT and JH) by 
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reading through the transcripts and independently identifying 
the main themes. Any differences were then discussed and 
a final coding scheme agreed, consisting of themes and sub- 
themes (where applicable). Interviews were coded by two 
experienced researchers trained in use of the coding scheme 
(LH and RC). Thirty percent of their coding was randomly 
checked (by JH). The purpose of this form of content analy- 
sis is to produce a numerically based descriptive summary; 
therefore, the frequency of main themes is presented to assist 
the reader to interpret findings. Equal weight was given to 
deviant/disconfirming responses and they are also included 
in the results tables and summaries. 

As part of the assessment of viability, peer observers 
recorded the time taken to complete their observational 
assessments (including preparation, observation, and com- 
pleting their assessment). Analysis was undertaken using 
IBM" SPSS* Statistics for Windows version 19 software 
(IBM Corp, Armonk, NY, USA). The protocol for this study 
was reviewed by the UK National Research Ethics Service 
and approved as a service development project. 

Results 

Usability of the observational tool 

All observers reported that the tool was easy to use: "It was 
easy to use, the instructions were clear" (manager, observer). 
Some provided ideas about further improvement of the tool, 
including reducing the rating scale and/or providing more 
detailed descriptive anchors for points on the rating scale 
to help ensure reliability across different time periods and 
observers, and to help the MDT interpret the observer's 
assessment (53%, 10/19); "Giving a score was sometimes 
difficult, you need a benchmark when rating and a scale of 
0-10 is too wide, I think a 4 or 5 point scale, with a written 
description of each anchor and guidance, would be good" 
(noncancer doctor, observer). 

Viability of observational 
assessment by peers 

Characteristics and experience of peer observers 

The 19 peer observers included seven managers, eight 
clinicians (four of whom also held managerial roles), and 
four MDT administrators (described as "cancer pathway 
coordinators"). The clinicians included six cancer consultant 
physicians, one noncancer consultant physician, and one 
senior allied health professional. Peer observers described 
their prior experience of MDMs as having either participated 
in MDMs as a member (63%, 12/19), or having worked with 
and/or observed MDMs as part of their job (37%, 7/19). 



None of the peer observers had previous experience of using 
an observational assessment tool. 

When asked for views regarding the importance of a 
professional group of peer observers, some responded that 
knowledge or experience of MDT working and ability to 
provide objective feedback was more important than a 
professional group per se (58%), 1 1/19), whilst others felt that 
a professional background was important, stating that a clini- 
cal viewpoint was necessary in order to judge appropriately 
all elements of MDT working, such as clinical decision- 
making (37%o, 7/19, three of whom were clinicians). All 
MDT administrators who undertook observations reported 
that peer observers needed to be more senior than themselves 
because they felt the MDT would be more likely to make 
changes if the feedback came from a more senior observer: 
"Although I was happy and felt able to do it, I think the 
observer should really be someone at a higher level (than me) 
to ensure that the MDT listens to what is said and makes the 
changes needed" (cancer pathway coordinator, observer). 

Observers' views about feasibility 
of observational assessment 

Two thirds (12/19) of the peer observers stated that it was 
feasible for them to undertake the assessments alongside 
their usual job, and the remainder said the main challenge 
was finding time due to other workload pressures. However, 
despite these challenges, two of these observers stated they 
would be happy to observe again, and a third suggested there 
was a learning curve that would make it easier next time: 
"This was the first time I had done anything like this, so there 
was probably more preparation, and it would be easier if I did 
it again" (cancer consultant doctor, peer observer). Only two 
observers (11%) said they would like to observe a second 
meeting to provide further validation of their assessments, 
but neither did this, stating that the main reason was not 
having sufficient time. The mean time reported to complete 
observational assessments (observing the MDM and writing 
their feedback), by the professional group was: 1 17 minutes 
for clinicians (range 90-135 minutes, including 89 minutes 
observing the MDM); 151 minutes for managers (range 
70-270 minutes, including 72 minutes observing the MDM); 
and 234 minutes for administrators (range 1 80-300 minutes, 
including 80 minutes observing the MDM). 

Views about utility of feedback 
from observational assessment 

Views of MDT members 

In total, 64 interviews were conducted with MDT members 
(32 consultant physicians and 14 nursing, 15 administrative, 
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and thi-ee allied health professionals). Of these, 25 had been 
observed in vivo, and 39 via a videorecorded meeting. Three 
quarters described the observational assessment and feedback 
they received as a useful process that provided valuable and 
representative feedback (73%, 47/64, Table 2), and that it 
had the potential to influence practice. As described by one 
surgeon: "It gives an opportunity to see where improvements 
are needed and where we are doing well. It was useful . . . 
we've already made some changes." 

A minority commented on the limitations of observa- 
tion, including whether the process of being observed alters 
"normal" behavior (ie, the observer effect) and whether 
it was possible to make valid assessments of the team on 
the basis of one observed meeting (20%, 13/64, Table 2). 



Seven MDT members expressed the view that observational 
assessment was not useful for these reasons. There were no 
apparent differences in MDT members' perceptions of the 
usefulness of peer assessment according to the professional 
background of the observer. 

Views of observers 

All observers felt that providing their observational feedback 
was useful to the MDT(s) they observed (and to themselves), 
and felt it could inform improvements in MDT performance 
(100%, 19/19). Two thirds of the peer observers were con- 
fident that their feedback was representative of the usual 
performance in the MDM they observed (68%, 13/19); 
a minority were more tentative, instead describing limitations 



Table 2 Views of MDT members and peer observers about the utility of observational assessment 



Themes 



Representative quotes 



Subthemes 



MDT members (number identifying theme/total 
number) 



Peer observers (number identifying 
theme/total number) 



Observational assessment 

and feedback useful 

Representative, valuable, 
influence on practice 
(facilitate change) 
Observational tool easy 
to use/feedback easy to 
interpret 



Limitations of observation 
Observation effect 
Concerns about validity/ 
representativeness 
Concerns about 
feedback consequences 
of providing feedback 



(47/64) 

"Independent observation provides an objective perspective. 
The feedback made everyone av/are that they seem to be 
doing things correctly from an external perspective. It gives 
an opportunity to see where improvements are needed 
and v/here we are doing well. It was useful . . . we've already 
made some changes" (surgeon) 

"We were aware of the presence of the observer but still the 
meeting felt natural. There was no resistance from team about 
observation, we're all quite used to being observed from 
medical students, new members of staff etc" (oncologist) 
"(Changing the room layout after reading observer's 
assessment) has already made difference and made the 
meeting feel different. It was useful that they highlight the 
importance of CNS (clinical nurse specialist) in discussions 
but also the AHP (allied health professional), we had 
become complacent. This change keeps us focused on 
meeting, we have less unrelated conversations" (nurse) 
"It's useful to get a new perspective from someone with 
no bias, it's changed the MDM already, and there is now 
more emphasis by team members to find out more about 
patients before proceeding with the meetings. It's raised 
our awareness" (MDT coordinator) 
(13/64) 

"Observation bring out two types of behavior, I) people 
become shy and quiet or 2) surgeons are more likely to 
be rebellious - so it's difficult to determine whether that 
behavior is natural or not" (radiologist) 
"The observation only picks up only what goes on inside 
meeting like the lack of input from nursing team, but they 
have already discussed patient with surgeon therefore 
surgeon expresses shared views - so there's no point in 
nurses repeating" (surgeon) 



(19/19) 

"Everyone is so busy and we don't get the 
time to step back and have a look what's 
going on" (manager) 

"It was easy to use and wasn't onerous to 
do, and actually quite really enjoyed it, it was 
really pleasing to see and it was good to be 
part of it" (manager) 

"Observing one meeting was enough because 
I have extensive experience in taking part 
in MDTs, so I was confident in judgments" 
(oncologist) 

"I think (the observational tool) was easy 
to use overall ... There was nothing I felt 
I couldn't comment on and it also gave you 
the free text boxes to comment on anything if 
and when you needed to" (oncologist) 



(6/19) 

"I felt like the team were on best behavior a 
bit" (obstetrician) 

"It would be beneficial to see more than one 
to make sure it was representative of the 
team, but I didn't have time" (manager) 
"I was thinking oh God I hope I haven't 
offended anyone and you know I do need to 
have a good working relationship with these 
people. I was honest but polite ... you have to 
think about your feedback because you might 
have to interact with these people" (manager) 



Abbreviations: MDM, muitidisciplinary meeting: MDT, multidisciplinary team. 
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such as the possibility of an observer effect and their own 
concerns about potential problems caused by providing 
negative feedback to MDTs within one's own workplace, 
although no such problems actually occurred during the 
study (Table 2). 

Discussion 

This study provides proof of concept that observational 
assessment and feedback by peers of the quality of cancer 
MDMs could be feasible and provide utility in relation 
to the appraisal of MDM performance. Further, although 
changes to team working went beyond the scope of this 
proof of concept study, most MDT members reported that 
peer assessment and feedback could facilitate improvements 
in MDM functioning. 

Most cancer service managers were able to identify 
and recruit clinicians or managers to act as peer observers. 
However, at one hospital, the observers (all experienced 
at MDT working but in an administrative role) viewed 
themselves as insufficiently senior for their assessment to 
have an impact on the team. The importance of seniority 
in observational assessment of health care MDTs has been 
highlighted in a recent study that used an expert consensus 
Delphi approach to determine the guidelines for team 
assessment (observation) in perioperative care. The expert 
guideline is that if observers are not experts in observational 
techniques (such as psychologists or specialists in human 
factors), they should be senior peers in order to have the 
necessary skills set.^' 

Observers generally felt confident in their ability to form 
a fair assessment fi-om observing just one meeting, although 
some MDT members and observers raised concerns about 
this. The possibility that there could have been a Hawthorn 
effect (ie, the presence of an observer or camera altering usual 
behavior) was raised by some MDT members and observers. 
This is a common cause of concern in observational studies,'^ 
although previous research with cancer MDTs has found 
that MDT members report that meetings are still "typical" 
despite the presence of an observer and/or video camera, 
with MDT members soon forgetting about the presence of 
these. Indeed, most MDT members we interviewed stated 
that observers' comments were representative of their general 
performance. Further, if such assessments become part of 
routine practice, it is likely that any impact on usual behavior 
would be minimized as MDT members become accustomed 
to observational assessment." 

The observational assessment tool developed for this study 
was based on "The Characteristics of an Effective MDT".^** 



Although developed within the context of the UK health 
service, it is likely that the ten domains identified as important 
for optimal functioning of MDMs (see Figure 2) would be 
equally relevant and important in cancer MDTs outside the 
UK. As highlighted by some observers, the ease of use of the 
tool could be improved if the size of the scale was reduced and 
if it incorporated a behaviorally anchored rating scale for the 
extremes and mid point of the rating scale; such an approach 
is common for performance measures^" and has been used by 
other observational assessment tools.^"* These changes should 
make the scale more user- friendly for peer observers, whilst 
not adversely effecting scale reliability;^'-"' further research 
is needed to develop these and then to evaluate the validity 
and reliability of the tool. 

Although structured observational assessment tools for 
use in cancer MDMs have been developed, they cannot be 
used easily without expert training and have a significant 
learning curve. ^''^^ This study suggests that peers with no 
formal training in observational techniques (but provided with 
short written guidance) can complete structured observational 
assessments using our study tool. A Cochrane review of 
49 randomized trials suggests that providing feedback can 
highlight areas for improvement and lead to improvements 
in health care professional practice.^' Similarly, the present 
study suggests that cancer MDT members felt that feedback 
would be useful to facilitate change. Previous work suggests 
that MDTs value feedback about their effectiveness based on 
self-assessment questionnaires,^' and future work is needed 
to determine the impact of feedback on objective changes to 
MDT working (and ultimately in relation to improvements 
in patient care). 

This study has some limitations. It was a small proof 
of concept study and its aims were exploratory; further, it 
needs to be replicated with a broader range of organizations, 
MDTs, and peer observers. Only subjective views about 
the usefulness of observational assessment and feedback to 
facilitate change were assessed, and not whether feedback 
led to actual change because time limitations meant we 
could only enquire about immediate change (not longer-term 
impact). The reliability and validity of scoring by assessors 
also needs to be further evaluated. The hospitals and MDTs 
were a convenience sample, which may have biased the 
findings, for example, by including MDTs more accepting 
of assessment compared with other MDTs. Further, as with 
all qualitative studies, our findings reflect the opinions of 
individuals within a specific context, therefore the aim is 
not generalizability. However, similar themes were evident 
from perceptions of MDT members and observers across all 
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hospitals, regardless of tumor type, professional group, or 
geographic location, and we specifically sought and presented 
any deviant/disconfirming findings. 

Conclusion 

Observational assessment and feedback by peers could 
provide a useful approach to MDT development. This 
proof of concept study suggests that observational 
assessment by peers could be a feasible and acceptable 
approach that may enhance MDT performance. With 
further validation, this could provide a useful means by 
which to help improve MDM functioning and share good 
practice. 
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